Image sensing devices, such as charge-coupled devices (CCDs), are commonly found in such products as digital cameras, scanners, and video cameras. These image sensing devices have a very limited dynamic range when compared to traditional negative film products. A typical image sensing device has a dynamic range of about 5 stops. As a consequence, the exposure level for a typical scene must be determined with a fair amount of accuracy in order to avoid clipping the signal. As defined herein, exposure level is the total amount of light allowed to fall on an image sensing device during the process of sensing a scene to produce an image. When sensing a scene under fixed illumination with an imaging system with an optical path that has a fixed aperture, the exposure level is controlled by setting the imaging system's exposure time (shutter speed). When sensing a scene with fixed illumination with an imaging system with an optical path that has a variable aperture, the exposure level is controlled by setting the imaging system's exposure time and aperture.
Often times the scene has a very wide dynamic range as a result of multiple illuminants (e.g., front-lit and back-lit portions of a scene). In the case of a wide dynamic range scene, choosing an appropriate exposure for the subject often necessitates clipping data in another part of the image. The narrower dynamic range of an image sensing device relative to a scene therefore results in lesser image quality for images obtained by an image sensing device.
Methods to increase the dynamic range of images acquired by an image sensing device would allow such images to be rebalanced to achieve a more pleasing rendition of the image. Also, images with high dynamic range would allow for more pleasing contrast improvements, such as described by Lee et al. in commonly assigned U.S. Pat. No. 5,012,333, entitled “Interactive dynamic range adjustment system for printing digital images.”
One method used for obtaining improved images with an image sensing device is exposure bracketing, whereby multiple still images of the same resolution are captured at a range of different exposure levels, and one of the images is selected as having a best overall exposure level. This technique, however, does not increase the dynamic range of any individual image captured by the image sensing device. As defined herein, the term resolution is used to refer to the number of pixels in an image.
One method for obtaining an image with a high dynamic range is by capturing multiple still images of the same resolution having different exposure levels, and then combining the images into a single output image having increased dynamic range. This approach is described commonly assigned U.S. Pat. No. 5,828,793 to Mann, entitled “Method and apparatus for producing digital images having extended dynamic ranges,” and by commonly assigned U.S. Pat. No. 6,040,858 to Ikeda, “Method and apparatus for expanding the dynamic range of sensed color images.” This approach often requires a separate capture mode and processing path in a digital camera. Additionally, the temporal proximity of the multiple captures is limited by the rate at which the images can be read out from the image sensor. Greater temporal disparity among captures increases the likelihood of motion existing among the captures, whether camera motion related to hand jitter, or scene motion resulting from objects moving within the scene. Motion increases the difficulty in merging multiple images into a single output image.
Another method for obtaining an image with high dynamic range which addresses the issue of motion existing among multiple images is the simultaneous capture of multiple images having different exposure levels. The images are subsequently combined into a single output image having increased dynamic range. This capture process can be achieved through the use of multiple imaging paths and sensors. However, this solution incurs extra cost due to the multiple imaging paths and sensors. It also introduces a correspondence problem among the multiple images, as the sensors are not co-located and thus generate images having different perspectives. Alternatively, a beam-splitter can be used to project incident light onto multiple sensors within a single image capture device. This solution incurs extra cost for the beam-splitter and multiple sensors, and also reduces the amount of light available to any individual image sensor thereby lessening the image quality because of a decrease in signal-to-noise performance.
Another method for obtaining an image with high dynamic range is through the use of an image sensor having some pixels with a standard response to light exposure and other pixels having a non-standard response to light exposure. Such a solution is described in commonly assigned U.S. Pat. No. 6,909,461 to Gallagher et al., entitled “Method and apparatus to extend the effective dynamic range of an image sensing device.” Such a sensor has inferior performance, however, for scenes having a narrow dynamic range, as the pixels with a photographically slower, non-standard response have poorer signal-to-noise performance than pixels with a standard response.
Another method for obtaining an image with high dynamic range is through the use of an image sensor programmed to read out and store pixels within the image sensor at a first exposure level while continuing to expose the image sensor to light. Such a solution is described in commonly assigned U.S. Pat. No. 7,616,256 to Ward et al., entitled “Multiple exposure methods and apparatus for electronic cameras.” In one example, pixels from a CCD are read into light-shielded vertical registers after a first exposure level is achieved, and exposure of the image sensor continues until a second exposure level is achieved. While this solution allows multiple readouts of individual pixels from the image sensor with minimal time between the exposures, it has the drawback of requiring specialized hardware to read the data out from the sensor.
Therefore, a need in the art exists for an improved solution to combining multiple images to form an image having high dynamic range, without requiring special hardware or additional image sensors, without sacrificing performance for scenes not requiring high dynamic range, without requiring a separate capture mode, and with minimal time between the multiple exposures.